Current methods of image zooming include digital zooming and optical zooming. In digital zooming, a picture is cropped to an area of interest and the pixels of that area are blown up. Although the area of interest is “zoomed in” the resolution is significantly less than the physical pixel resolution of the image sensor. An alternative to digital zooming is optical zooming, where a motor moves the optical elements of the camera lens to change the lens' focal length, i.e. the distance between the center of the lens and the image sensor. Although optical zooming samples the actual zoomed image by the physical resolution of the image, its hardware requirements are ill-suited for miniature cameras such as those found on mobile devices.
In practice, super-resolution techniques may be applied to achieve higher resolutions when performing a digital zooms. Super-resolution can refer to various image processing techniques that construct a high-resolution (HR) image of a scene from multiple low-resolution (LR) images of the same scene. During image capture, different LR images of the same scene are taken by slightly moving the optical path to the image sensor after each image capture. Subsequently, a HR image (i.e., a super-resolution image) is constructed by combining information from a reference LR image with the information from the other, shifted LR images.
A critical step in the super-resolution reconstruction of images is image registration. During image registration the motion parameters of the LR images are estimated, and the LR images are then geometrically aligned. To achieve accurate super-resolution image reconstruction, image alignment must be precise. Accordingly, the availability of accurate registration parameters between the multiple LR input images is critical for achieving good performance in image super-resolution. Generally speaking, registration with a subpixel accuracy is necessary for the success of the reconstruction process.
Another important consideration in super-resolution reconstruction is motion blur. It is desirable to avoid blurring caused by the motion of the camera during image capture. If the amount of motion blur approaches or exceeds one pixel, little resolution enhancement, if any, can be made to a reference LR image. Motion blur is a particular concern for cameras used in handheld mobile devices.
Current implementations of super-resolution techniques in handheld mobile devices rely on optical image stabilization (OIS) along two orthogonal axes: pitch and yaw. The 2-axis OIS attempts to eliminate the image blur caused by the motion of the camera. Additionally, current super-resolution techniques implemented in mobile devices rely on the OIS itself to command movement of an actuator to cause specific subpixel movements of the optical path during image sampling.